10 research outputs found
A generative model for sparse, evolving digraphs
Generating graphs that are similar to real ones is an open problem, while the
similarity notion is quite elusive and hard to formalize. In this paper, we
focus on sparse digraphs and propose SDG, an algorithm that aims at generating
graphs similar to real ones. Since real graphs are evolving and this evolution
is important to study in order to understand the underlying dynamical system,
we tackle the problem of generating series of graphs. We propose SEDGE, an
algorithm meant to generate series of graphs similar to a real series. SEDGE is
an extension of SDG. We consider graphs that are representations of software
programs and show experimentally that our approach outperforms other existing
approaches. Experiments show the performance of both algorithms
Pareto Actor-Critic for Equilibrium Selection in Multi-Agent Reinforcement Learning
This work focuses on equilibrium selection in no-conflict multi-agent games,
where we specifically study the problem of selecting a Pareto-optimal
equilibrium among several existing equilibria. It has been shown that many
state-of-the-art multi-agent reinforcement learning (MARL) algorithms are prone
to converging to Pareto-dominated equilibria due to the uncertainty each agent
has about the policy of the other agents during training. To address
sub-optimal equilibrium selection, we propose Pareto Actor-Critic (Pareto-AC),
which is an actor-critic algorithm that utilises a simple property of
no-conflict games (a superset of cooperative games): the Pareto-optimal
equilibrium in a no-conflict game maximises the returns of all agents and
therefore is the preferred outcome for all agents. We evaluate Pareto-AC in a
diverse set of multi-agent games and show that it converges to higher episodic
returns compared to seven state-of-the-art MARL algorithms and that it
successfully converges to a Pareto-optimal equilibrium in a range of matrix
games. Finally, we propose PACDCG, a graph neural network extension of
Pareto-AC which is shown to efficiently scale in games with a large number of
agents.Comment: 20 pages, 12 figure
Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workers
We envision a warehouse in which dozens of mobile robots and human pickers
work together to collect and deliver items within the warehouse. The
fundamental problem we tackle, called the order-picking problem, is how these
worker agents must coordinate their movement and actions in the warehouse to
maximise performance (e.g. order throughput). Established industry methods
using heuristic approaches require large engineering efforts to optimise for
innately variable warehouse configurations. In contrast, multi-agent
reinforcement learning (MARL) can be flexibly applied to diverse warehouse
configurations (e.g. size, layout, number/types of workers, item replenishment
frequency), as the agents learn through experience how to optimally cooperate
with one another. We develop hierarchical MARL algorithms in which a manager
assigns goals to worker agents, and the policies of the manager and workers are
co-trained toward maximising a global objective (e.g. pick rate). Our
hierarchical algorithms achieve significant gains in sample efficiency and
overall pick rates over baseline MARL algorithms in diverse warehouse
configurations, and substantially outperform two established industry
heuristics for order-picking systems
Deep Reinforcement Learning for Multi-Agent Interaction
The development of autonomous agents which can interact with other agents to
accomplish a given task is a core area of research in artificial intelligence
and machine learning. Towards this goal, the Autonomous Agents Research Group
develops novel machine learning algorithms for autonomous systems control, with
a specific focus on deep reinforcement learning and multi-agent reinforcement
learning. Research problems include scalable learning of coordinated agent
policies and inter-agent communication; reasoning about the behaviours, goals,
and composition of other agents from limited observations; and sample-efficient
learning based on intrinsic motivation, curriculum learning, causal inference,
and representation learning. This article provides a broad overview of the
ongoing research portfolio of the group and discusses open problems for future
directions.Comment: Published in AI Communications Special Issue on Multi-Agent Systems
Research in the U